1. Field of the Invention
The present invention relates to diagnostic tools for analyzing software. More particularly, the invention is directed to the dynamic instrumentation of running software in order to gather runtime data as the software executes. Still more particularly, the invention concerns a technique for implementing a just-in-time dynamic instrumentation.
2. Description of the Prior Art
By way of background, in order to analyze the operation of a complex system such as a large software program, it is important to be able to gather operational information while the system is running, without halting execution, modifying the system, and then restarting it. One approach is to dynamically add instrumentation code to the running system when data is needed, so that there is no performance penalty. This is in contrast to statically instrumenting the system when it is created, which offers no flexibility. Operating systems are examples of complex systems that may require diagnosis at run time, as are many user-space applications, such as database management programs. As shown in FIG. 1A, it is often desirable to probe some software function to determine information about its operation, such as how often or why the function is being called. The software memory location where the function is to be probed is referred to as the probe point. As shown in FIG. 1B, dynamic instrumentation may be implemented by adding a software interrupt (e.g., an Int 3 breakpoint in x86 processors) (the probe) at the probe point in the running system and providing a probe handler that is executed when the interrupt throws an exception. The probe handler executes instrumentation code (an instrumentation function) that obtains the required information and provides it to the probe user, then returns control to the running system. Examples of such conventional dynamic instrumentation software include the Linux® System Tap framework. This framework provides a Linux® tool that allows application developers and system administrators to write probe scripts that will be automatically parsed, elaborated upon, translated, compiled and loaded as a kernel module to generate probe data from a running operating system kernel. Probing of user-space programs is also supported.
The manner in which conventional dynamic instrumentation solutions are invoked so as to become operational in a running software system has a number of disadvantages. For example, when dynamic instrumentation is implemented using an operating system kernel module, starting the instrumentation probe requires loading the kernel module and setting the breakpoint. Stopping the probe requires removing the breakpoint and unloading the module. This is not a practical way to activate dynamic instrumentation. Among other things, there is a delay associated with starting and stopping the instrumentation probe. Moreover, root privileges are typically required to load and unload the module, and there is no known programmatic way to turn the instrumentation on and off from a user-space application. This means that such applications cannot take advantage of this type of dynamic instrumentation. Note that it is impractical to pre-load the instrumentation kernel module because loading the module automatically activates the instrumentation, which should not run during normal operations (when probing is not being performed) due to the overhead associated with the instrumentation code.
Another conventional instrumentation approach places conditional static probes in the program code so that the probes only execute when instrumentation is desired. However, the probes alter the execution flow of a program on a permanent basis and add performance overhead by inserting a condition checking code path into the target program being instrumented, as follows:
if (probing);
do some probing work.
By default, when probing is not active the above “if” condition fails and the probing code is not active. When probing needs to be activated, the “if” condition variable is made true. This technique imposes a performance penalty due to the condition processing of the “if” statement.
Accordingly, it would be desirable to provide a techinque whereby a probe handler providing dynamic instrumentation can be easily activated and deactivated. What is needed is a technique that facilitates just-in-time dynamic instrumentation that does not have the overhead associated with conventional techniques. The ability to activate and deactivate dynamic instrumentation from user-space applications would be of further benefit. It would be additionally advantageous if instrumentation probes could be assigned to groups that can be specified when activating and deactivating instrumentation probes.